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Abstract 
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There  has  been  considerable  work  recently  on  the  problem  of  reconstruction  of  3D  point 
sets  from  two  images,  taken  by  uncalibrated  cameras.  However,  the  point  correspondence  has 
to  be  given.  Here  we  deal  with  reconstruction  of  curves  rather  than  points.  While  we  need 
the  correspondence  between  curves,  this  is  an  easier  problem  because  curves  are  far  fewer 
and  more  distinctive  than  points.  We  derive  a  simple  and  general  reconstruction  method, 
based  on  an  invariant  coordinate  system.  We  then  apply  it  to  non-coplanar  conics  and  to 
combinations  of  a  3D  conic  with  points.  3D  cubics  are  also  discussed.  Unlike  previous  work, 
we  do  not  need  to  know  the  epipolar  geometry;  we  recover  it  from  the  images. 
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1 9960408  1 40 


The  author  is  grateful  for  the  support  of  the  Air  Force  Office  of  Scientific  Research  under  grant  F49620- 
92-J-0332,  the  Defense  Advanced  Research  Projects  Agency  (ARPA  Order  No.  C635)  and  the  Office  of  Naval 
Research  under  Grant  N00014-95- 1-0521. 


1.  Introduction 


The  problem  of  3D  reconstruction  from  uncalibrated  cameras  is  closely  related  to  the  prob¬ 
lem  of  finding  projective  and  affine  invariants.  This  is  because,  regardless  of  the  camera 
parameters,  we  can  assume  that  the  camera  performs  a  simple  geometric  projection  (assum¬ 
ing  an  aberration-free  lens).  The  invariants  of  this  projection  are  thus  independent  of  the 
camera  parameters.  The  development  of  applications  of  invariants  in  vision  generally  (e.g. 
[13-15])  has  thus  led  to  the  development  of  such  applications  for  uncalibrated  cameras,  e.g. 
11-12], 

There  are  no  invariants  of  a  projection  from  3D  to  a  single  2D  image,  so  we  need  at  least 
two  views.  Given  the  projections  of  the  same  3D  object  in  the  two  images,  the  object  can 
be  reconstructed.  In  most  of  the  above  mentioned  work,  the  objects  are  points.  One  needs 
at  least  seven  corresponding  points  for  the  reconstruction,  or  eight  if  the  problem  is  to  be 
solvable  linearly.  Finding  point  correspondences  is  far  from  a  solved  problem  in  vision,  and 
this  limits  the  usefulness  of  these  methods. 

The  objects  we  deal  with  in  this  paper  are  curves  rather  than  points.  Finding  correspon¬ 
dence  between  curves  is  much  easier  than  between  points.  There  axe  far  fewer  curves  in  an 
image  than  there  axe  points,  so  a  combinatorial  explosion  is  unlikely  to  be  a  problem.  In 
addition,  unlike  points,  curves  have  features  that  distinguish  one  from  another  and  can  be 
used  to  narrow  down  the  possible  matches.  A  few  of  the  references  above  deal  with  conic 
curves  but  require  a  priori  knowledge  of  the  epipolar  geometry,  or  the  matching  properties 
of  the  images.  Here  we  recover  this  information  from  the  images. 

It  is  easy  to  calculate  the  minimal  number  of  parameters  the  object’s  description  has  to 
have  for  the  problem  to  be  solvable.  In  the  general  projective  case,  each  camera  projection 
from  3D  to  2D  is  a  3  x4  matrix,  having  11  essential  coefficients.  If  the  object  has  n  parameters 
in  a  known  coordinate  system,  we  have  n-1-22  unknowns.  Of  these,  we  give  up  15  by  using  an 
indeterminate  3D  coordinate  system  which  we  cannot  recover.  This  is  a  result  of  not  knowing 
the  camera  parameters.  Thus  we  need  to  find  n  +  7  unknowns.  To  solve  this  problem  we 
thus  need  at  least  n  -f-  7  known  quantities.  If  the  projection  of  the  object  has  m  measurable 
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parameters  in  each  image,  we  must  thus  have 


2m  >  n  +  7 

In  the  affine  case,  the  projection  matrix  has  8  parameters  and  the  3D  coordinate  system  has 
12,  so  similar  arguments  yield 

2m  >  n  +  4 

We  see  immediately  that  seven  3D  points  satisfy  the  above  formula  in  the  projective  case, 
with  a  total  number  of  n  =  21  parameters  in  3D  and  m  =  14  in  the  2D  projections.  In  the 
affine  case  four  points  suffice.  (However,  due  to  noise,  many  more  corresponding  points  in 
the  images  are  actually  needed.)  A  set  of  two  conics  in  3D  has  16  parameters,  projected  into 
ten  on  each  image.  Thus  such  a  set  will  satisfy  our  requirement  in  the  affine  case,  but  will 
leave  three  parameters  unknown  in  the  projective  case.  A  cubic  curve  has  12  parameters  in 
3D,  projected  into  eight  parameters  in  2D;  thus  it  too  will  satisfy  the  affine  requirement  but 
will  leave  three  parameters  unknown  in  the  projective  case. 

For  recognizing  the  objects,  only  some  of  the  quantities  solved  for  will  be  useful.  These 
will  be  object  descriptors  which  are  invariant  to  a  3D  transformation.  (The  rest  of  the 
quantities  will  be  related  to  the  cameras).  An  object  with  n  parameters  will  have  n  —  15 
invariants  in  the  projective  case  and  rz  —  12  in  the  affine  case.  The  pair  of  3D  conics  thus 
has  one  invariant  in  the  projective  case  and  four  in  the  affine  case.  The  cubic  has  no  3D 
invariants  and  therefore  cannot  be  recognized.  In  other  words,  all  3D  cubics  are  affine 
(and  projective)  equivalents;  but  we  can  still  derive  some  information  from  them  about  the 
cameras  and  the  correspondence  between  the  two  images.  Higher  order  curves  can  in  theory 
also  be  reconstructed,  but  they  are  difficult  to  extract  from  the  images.  A  more  practical 
approach  for  a  general  curve  can  be  to  approximate  it  as  a  series  of  conic  pieces. 

Combinations  of  curves  and  points  are  edso  of  interest.  For  instance,  one  conic  and  two 
points  in  3D  have  two  invariants  in  the  affine  case. 

In  this  paper  we  solve  the  reconstruction  problem  in  two  stages: 

1)  We  recover  the  epipolar  geometry,  namely  the  matching  properties  between  the  two 
images. 
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2)  We  find  the  3D  invariants  of  the  objects,  based  on  the  matched  images.  This  is  done 
in  an  invariant  coordinate  system  which  makes  the  process  very  simple  and  general. 

2.  Finding  the  Epipolar  Geometry 

In  this  section  we  find  the  epipolar  geometry,  namely  the  matching  properties  between  the 
images.  This  will  enable  us  to  reconstruct  the  objects  in  the  next  section. 

We  have  two  cameras  with  optical  centers  at  0,0'  in  3D.  The  line  connecting  these 
points  intersects  the  first  image  at  o  and  the  second  image  at  o'.  These  are  the  epipoles. 
In  other  words,  an  epipole  is  the  projection,  in  one  camera’s  image,  of  the  other  camera’s 
optical  center.  An  arbitrary  point  X  in  3D  defines  an  epipolar  plane  OO'X  going  through 
it  and  the  optical  centers.  This  plane  intersects  the  first  image  in  a  line  ox  and  the  second 
image  in  a  line  o'x'.  These  are  the  epipolar  lines.  They  form  in  each  image  a  pencil  of  lines 
meeting  at  the  epipole.  Obviously,  for  a  given  X,  the  epipolar  lines  match  each  other  since 
they  belong  to  the  same  plane.  Finding  the  epipolar  geometry  means  finding  the  points  o,  o' 
and  finding  the  matching  between  the  epipolar  lines. 

The  epipolar  geometry  can  be  succinctly  represented  by  the  so  called  fundamental  matrix. 
In  the  case  of  points,  it  is  quite  easy  to  find  this  matrix,  given  the  point  correspondence.  In 
our  case  of  curves,  we  did  not  find  this  matrix  particularly  useful;  we  therefore  use  a  different 
method. 

Our  method  is  based  on  invariants  of  the  epipolar  lines  that  are  equal  in  the  two  images. 
These  can  be  found  as  follows.  The  points  X,-  form  planes  OO'Xj  having  a  common  line 
OO'.  Thus  we  can  define  a  cross  ratio  of  these  planes,  in  the  projective  case,  or  a  ratio  in 
the  affine  case.  These  planes  intersect  the  first  image  in  the  epipolax  lines  ox,-,  and  therefore 
these  lines  have  the  same  (cross)  ratio  as  the  planes  on  which  they  lie.  Similarly,  in  the 
other  image,  the  matching  epipolar  lines  o'x'  have  the  same  (cross)  ratio  as  their  planes. 
Therefore,  the  (cross)  ratio  of  matching  epipolar  lines  is  the  same  in  both  images. 

This  invariance  can  be  used  to  find  the  epipoles  as  follows.  Given  some  matching  points 
Xi,x(,  we  can  express  the  epipolar  lines  going  through  them  as  functions  of  the  unknown 
epipoles  o,o',  since  these  lines  go  through  the  epipoles.  By  equating  the  ratios  of  matching 
epipolar  lines,  we  obtain  enough  equations  to  determine  o,  o'.  For  matching  curves,  it  is  easy 
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to  find  matching  epipolar  lines  by  drawing  tangents  to  the  curves  from  the  epipoles.  Since 
these  tangents  are  projections  of  the  corresponding  tangent  planes  to  the  3D  curves,  they 
will  match.  The  contact  points  of  the  tangents  with  the  curves  will  also  match.  We  can  then 
use  the  method  described  above,  i.e.  use  the  ratios  of  these  tangents,  to  write  equations  for 
finding  the  epipoles. 

In  the  affine  case,  the  epipoles  axe  at  infinity,  meaning  that  all  epipolar  lines  in  each 
image  are  parallel.  Thus  all  epipolar  lines  in  the  first  image  have  the  same  slope  s,  while 
those  in  the  second  image  have  a  slope  5'.  The  problem  of  finding  the  epipoles  is  now  reduced 
to  finding  the  slopes  s,  s'. 

We  will  now  perform  the  calculation  in  the  affine  case  for  various  configurations.  We 
can  represent  a  family  of  parallel  lines  having  a  common  slope  s  as  a  vector  in  homogeneous 
coordinates 

l  =  (s,l,<J) 

where  d  is  a  distance  parameter  proportional  to  the  intercept. 

Given  three  epipolar  lines  with  different  d,,  we  can  define  their  ratio  as  (di  -  d2)/ (ds  - d2). 
This  is  an  affine  invariant  and  is  the  same  in  both  images  if  the  lines  match.  Given  four  such 
lines,  we  have  two  invariant  ratios,  which  will  yield  two  equations  for  s,s'. 

2.1.  Two  Conics 

Two  conics  in  an  image  have  four  tangents  with  the  same  slope  s,  so  they  can  be  used  in 
our  method. 

The  tangents  to  a  conic  satisfy  the  equation  of  the  line  conic,  the  dual  of  the  point  conic. 
Denoting  the  regular  (point)  conic  by  the  matrix  A  the  line  conic  is  A  =  aij. 

To  find  tangents  with  slope  s,  we  substitute  the  line  vector  in  the  line  conic: 


fs\ 

lAl'  =  (5,l,d) 

a2l  <222  ^23 

1- 

\  ^^31  <^32  1  / 

[dj 

From  the  symmetry  of  A  we  obtain 

-j-  2cii2<s  +  022  "h  +  ‘lci2zd  +  d  =  0 
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The  two  solutions  for  this  quadratic  equation  in  d  are 


^1,2  =  ~®23  -  0135  ±  y  (Oia  -  aii)s2  +  2(013023  “  012)5  +  O^g  -  O22 

To  get  rid  of  the  square  root  it  is  more  convenient  to  deal  with  the  “width”  of  the  conic, 
d\  —  dl,  and  with  the  d  of  the  line  going  through  its  center,  =  |(dj  —  d^): 

-(dj  —  d^)^  =  (oi3  —  aii)s^  +  2(013023  ~  012)5  +  O23  —  O22 

=  ^(<^1  +  ^2?  =  -023  -  0135 
Similarly,  for  the  other  line  conic  B  we  have 

\{A  -  d\f  =  (bl  -  bn)s^  +  2(613623  -  612)5  +  6^  -  622 
dl=\{dl  +  dlf  =  -b,s-b^ss 

We  make  the  expressions  for  d“,  d^  independent  of  the  unknown  s  by  choosing  a  coordinate 
system  in  which  013  =  613  =  0.  This  is  equivalent  to  choosing  a  y-axis  that  goes  through  the 
centers  of  both  conics. 

From  the  four  values  of  d  we  can  obtain  two  affine  invariant  ratios: 


(dl  -  dlf 

“h  2ai2-5  —  CI23  +  0^22 

{di  -  dif  - 

(a23  —  623)^ 

{4  -  4? 

6ii5^  +  2612S  —  623  +  622 

(4  -  4r 

(023  —  623)^ 

In  the  other  image,  we  have  again  two  (line)  conics  A'^B'.  The  epipolar  lines  have  a 
slope  s'  with  respect  to  the  local  coordinate  system.  Therefore  two  invariants  similar  to 
those  above  exist,  with  b[j,s'  replacing  the  primeless  quantities  above. 

As  discussed  before,  these  invariant  ratios  axe  the  same  in  both  images,  i.e. 


djiS^  “f*  2(1i2S  —  <223 

+  022 

2<2j^2^  ^23  ^22 

(1) 

(023  —  623)^ 

(023 — 633)^ 

6iis^  +  2612S  —  623 

+  622 

b[^s'^  +  2¥,,s'-b'a^  +  V^ 

(2) 

(ot23  —  623)^ 

(O23  —  623)^ 

We  have  thus  obtained  a  system  of  two  quadratic  equations  for  the  two  unknowns  s,  s'.  They 
can  be  solved  by  standard  elimination  methods.  We  obtain  a  quartic  equation  for  s  (or  s'), 
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leading  to  four  solutions  to  the  problem.  As  in  the  case  of  points,  extra  information  such  as 
more  matching  conics  reduces  the  problem  to  a  system  of  linear  equations. 

After  finding  the  slopes,  it  is  easy  to  find  all  the  matches  between  epipolar  lines.  Two 
epipolar  lines  with  parameters  d,d'  match  each  other  if  they  are  related  by  the  invariant 
ratio 

jd-dt)  _  {d'-df) 

{d-dl)  (d'-d^') 

with  all  other  quantities  above  now  being  known. 

2.2.  A  Conic  and  Points 

The  method  described  above  is  easy  to  extend  to  a  combination  of  a  conic  and  points.  Given 
points  Xj',  they  lie  on  the  epipolar  lines  (s,  l,d,),  i.e.  they  satisfy 

liXi  =  sxi  +  yi  +  di  =  0 

We  thus  have  the  following  equations  for  d,,  d'f 

di  —  sXj  yj,  d^  s  Xj 


We  can  now  define  the  two  ratios 


1  {dl  —  dlY  _  (a^3  —  aii)s^  +  2(013023  -  <^12)^  +  a\z  ~  Q22 

2  (di  -  ^2)2  “  (-5(xi  -  X2)  -  (yi  -  y2)f 


d“  —023  ~  QlS-S 


dx  -  d2  -s{xx  -  X2)  -  {yx  -  ya) 

We  can  choose  a  coordinate  system  in  which  xi  — X2  =  0,  eliminating  s  from  the  denominator. 
Equating  the  above  two  ratios  to  the  ones  in  the  other  image  we  obtain 

(<^13  “  +  2(013023  ”■  012)5  +  023  ~  ^22  _  {^'xz  ~  2(Oi3Q23  ~  ^12)^  ^23  ~  ^22 

(y2  -  yxY  (y'z  -y'lY 


—  Cl2Z  ~  O13S 

y2-yx 


-Q23  - 

y2-yx 


We  have  two  equations  for  s,  s' ,  which  are  easy  to  convert  to  one  quadratic  equation. 
Given  an  additional  point,  we  obtain  an  additional  linear  equation: 

-5(xi  -  X3)  -  (yi  -  yz)  _  -5'(xi  -  X3)  -  (y(  -  y'^) 


y2  -yi 


y2-yx 
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Eqs.  (4), (5)  are  a  linear  system  in  s,s'.  They  are  equivalent  to  a  four-point  set,  with  the 
conic  center  (an  affine  invariant)  being  the  fourth  point.  Given  a  system  of  many  points 
and  conics,  we  can  solve  an  overdetermined  linear  system  of  equations  either  for  s,  s'  or  for 
s,s',  to  obtain  a  more  robust  solution. 

2.3.  A  Cubic 

A  cubic  curve  in  3D  can  be  written  in  homogeneous  coordinates  as 

X(r)  =  Co  +  CiT  -H 

with  C,  being  constant  vectors.  It  is  projected  into  the  images  as  two  cubic  curves  x(t),  x'(t'). 
The  parameters  in  3D  and  in  the  two  images  do  not  necessarily  match.  In  fact,  each  param¬ 
eter  can  change  with  three  degrees  of  freedom,  while  the  curve  remains  a  cubic: 

at  +  b 

t  = - ^ — 

ct-{-  d 

If  the  parameters  t ,  at  three  points  are  known,  the  three  constants  a,  b,  c  can  be  found  and 
this  determines  the  parameterization  uniquely.  Thus,  given  two  images  of  a  cubic  with  three 
matching  points,  the  matching  of  any  other  point  on  the  cubic  is  determined.  (This  in  in 
fact  true  for  any  rational  curve). 

To  find  three  matching  points,  we  find  three  tangents  to  the  cubic  having  slope  s  in  the 
first  image  and  s'  in  the  other.  Solving  a  cubic  equation  for  the  tangents,  we  obtain  three 
values  di{s),  d((s')  in  each  image.  These  values  obviously  depend  on  the  unknowns  s,  s'. 

The  contact  points  of  these  tangents  match,  and  so  they  can  be  assigned  the  same  values 
ii  in  both  images,  say  0,1, oo.  This  will  determine  the  coefficients  a,b,c  for  each  image, 
and  thus  establish  the  same  unique  parameterization  in  both  images.  This  solves  the  point 
correspondence  problem  between  the  two  images. 

To  find  the  epipolar  geometry,  we  can  now  select  a  fourth  point,  say  t  =  2,  in  each  image 
and  find  the  line  with  slope  s  (s')  going  through  this  point  in  the  first  (second)  image.  We 
have  thus  obtained  four  matching  parallel  lines  in  each  image,  depending  on  s,s'.  As  in  the 
conic  case,  we  have  two  invariant  ratios,  providing  two  equations  for  the  unknowns  s,s'. 
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Higher-order  curves  can  be  dealt  with  in  a  similar  way,  using  three  of  their  n  matching 
tangents  to  establish  the  parameter  correspondence.  However,  these  tangents  would  not  be 
very  reliable  in  practice. 

3.  Reconstruction 

Having  found  the  epipolar  geometry,  we  can  reconstruct  the  3D  object. 

3.1.  Invariant  Coordinates 

The  reconstruction  is  easier  in  term  of  a  coordinate  system  which  is  common  to  the  2D  and 
3D  spaces  and  is  invariant  in  both. 

We  build  this  coordinate  system  using  a  set  of  reference  points  which  axe  determined 
invariantly  by  the  3D  object  and  its  projections.  We  need  four  reference  points  in  3D,  and 
three  of  their  projections  in  each  image.  In  the  case  of  two  3D  conics,  we  can  use  in  3D  the 
four  contact  points  X,-  of  the  conics  with  the  epipolar  planes  tangent  to  them.  These  X,-  are 
projected  onto  the  images  as  the  contact  points  of  the  conics  with  the  epipolar  lines  tangent 
to  them.  These  tangents  have  known  slopes  s,  s'  and  they  touch  the  conics  at  known  points 
Xj,x^.  These  reference  points,  both  in  3D  and  2D,  are  thus  determined  invariantly. 

We  can  assign  to  the  reference  points  in  3D  the  standard  coordinates 

(1, 0,0,1),  (0,1,0,1),  (0,0,1,1),  (1,1, 1,1)  (6) 

In  2D  we  need  only  three  points  in  each  image.  We  can  use  Xi,X2,  X4  in  the  first  image  and 
a  slightly  different  subset,  say  xj,X3,X4,  in  the  other  image.  Both  subsets  can  be  assigned 
the  coordinates 

(1,0,1),  (0,1,1),  (1,1,1)  (7) 

in  their  respective  images.  These  choices  make  the  projection  matrices  from  3D  to  2D  very 
simple  in  this  “canonical” ,  invariant  system; 


(1 

0 

0 

ON 

(1 

0 

0 

ON 

p  = 

0 

1 

0 

0 

P'  = 

0 

0 

1 

0 

u 

0 

0 

ij 

u 

0 

0 

1/ 

Using  this,  the  remaining  3D  point  is  projected  into  the  origin  (0,0,1)  in  each  image. 
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Any  point  in  3D  can  be  expressed  in  terms  of  our  3D  reference  points  as 


U  =  {U,V,W) 

while  in  2D  we  have 

u  =  (u,  u),  u'  =  {u',  v') 

Using  the  projection  matrices  we  find  a  very  simple  relation  between  the  3D  and  the  2D 
invariant  coordinates: 

u  =  u'  =  U,  u  =  U,  v'  =  W  (8) 

That  is,  the  first  coordinate  is  common  to  both  images  and  to  the  3D  space.  It  is  related  to 
the  invariant  ratio  used  earlier  to  find  the  epipolar  geometry.  The  other  coordinates  are  also 
common  to  2D  and  3D  and  can  also  be  easily  interpreted  geometrically  as  ratios  of  planes 
and  lines  common  to  3D  and  2D. 

The  reconstruction  can  now  proceed  in  two  steps. 

1)  Given  the  curves  in  2D  Cartesian  coordinates  x,  x'  in  the  images,  transform  them  to  the 
invariant  coordinates  u,  u'.  The  transformation  matrices  between  the  two  coordinate 
systems  can  be  easily  found  from  the  linear  equations 

Xi  =  jVLi,  X-  =  fu'i 

where  Ui,u'-  are  the  invariant  reference  points  as  expressed  in  the  invariant  system, 
eq.  (7),  and  j,j'  are  the  transformation  matrices.  The  coordinates  x,x'  in  the  curve 
equations  are  then  replaced  by  jfu,_7'u'  to  obtain  the  curve  equations  in  terms  of  u,  u'. 

2)  Using  the  relations  (8),  the  2D  invariant  coordinates  in  the  curve  equations  are  replaced 
by  the  3D  ones.  This  creates  3D  (general)  cylinders  arising  from  each  image.  These 
cylinders  intersect  to  form  our  3D  curve. 

There  is  also  a  3D  transformation  matrix  J  analogous  to  the  2D  ones,  satisfying 

Xi  =  JVi 

with  U,  being  the  3D  reference  points  as  expressed  in  the  invariant  systeiri,  eq.  (6).  However, 
since  the  X,  are  unknown,  we  cannot  find  J .  This  matrix  contains  the  unknown  camera 
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parameters.  Therefore,  we  can  reconstruct  the  3D  object  only  up  to  an  arbitrary  affine 
transformation  J. 

The  projection  matrices  P,  P'  can  be  found  from  the  “canonical”  ones  P,  P'  defined  earlier 

by 

P  =  jPJ-\  P'  =  j'P'J-^ 

Thus,  if  some  of  the  coefficients  in  P,P'  are  known,  we  can  recover  some  of  the  coefficients 
in  J  and  therefore  have  more  information  about  the  3D  object. 

3,2.  Reconstructing  Conics 

We  will  now  find  the  3D  conic  equations  in  invariant  3D  coordinates. 

Carrying  out  the  first  step  above,  we  move  the  conics  in  the  images  to  the  2D  invariant 
system.  We  have  the  (point)  conics  A,  P  in  one  image  and  A',B'  in  the  other,  satisfying 

x*Ax  =  0,  x'^AV  =  0 


and  similarly  for  P,  P'.  Replacing  x  by  jxi  and  x'  by  ju'  we  obtain  the  conic  matrices  in  the 
invariant  system 

A  =  j^Aj,  A'  =  fA'f 

which  satisfy  the  equations 

U^Au  =  Gulp  +  ^22^^  A  2012^17  +  2ai3U  A  2<223U  +  033  =  0 

u^*A'u'  =  A  ^22^^^  A  2a'i2uv'  A  2dj3U  A  2033^'  +  O33  =  0 

and  similarly  for  P,  P'.  The  coefficients  ai,d(  are  not  independent.  There  is  only  one 
independent  quantity  for  each  2D  conic.  This  is  because  in  the  invariant  coordinate  system, 
the  contact  points  of  the  conic  with  the  epipolar  tangents  have  fixed  coordinates,  namely 
eq.  (7)  and  (0,0,1).  These  also  determine  the  slopes  s,s'  as  fixed  values.  The  conic  has  to 
pass  through  two  of  these  contact  points  and  be  tangent  there  to  the  epipolar  lines.  This 
determines  four  out  of  the  five  conic  parameters.  Incidentally,  one  of  the  contact  points  in 
the  invariant  system  is  the  origin,  therefore  one  of  the  coefficients  033,  ^33  equals  0,  while  the 
other  equals  1. 
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Carrying  out  the  second  step,  we  replace  the  2D  coordinates  u,v,v'  in  the  equations 
above  by  the  3D  coordinates  U,V,W  using  eq.  (8).  We  first  deal  with  the  conics  A,  A'.  We 
obtain 


+  Ct22^^  A  2cti2UV  -f-  ‘IcixzU  A  2G23^  a  ®33  =  0 
+  2a[^UW  A  2a'^^U  +  2a'23W  +  033  =  0 

These  are  two  (conic)  cylinders,  the  axis  of  each  being  parallel  to  the  direction  of  the  pro¬ 
jection  onto  the  corresponding  image.  The  intersection  of  these  two  cylinders  forms  our  3D 
conic.  In  general,  two  cylinders  do  not  intersect  in  a  plane  conic.  However,  we  have  assumed 
a  priori  that  our  two  images  A,  A'  are  formed  by  the  same  3D  plane  conic,  so  this  conic  must 
lie  on  both  cylinders. 

We  will  now  find  the  intersection  of  the  cylinders.  In  general,  it  can  be  found  by  elimi¬ 
nation  methods.  However,  our  case  is  particularly  simple  because  the  intersection  is  planar 
and  the  planes  contain  the  3D  reference  points  (6).  A  plane  can  be  generally  written  in  the 
invariant  coordinate  system  as 


U  =  aiV  +  a2W  -t-  0:3 

Without  loss  of  generality,  we  can  assume  that  the  plane  H®,  containing  the  conic  arising 
from  A,  A\  also  contains  the  reference  points  (1,0, 0, 1),  (0, 1, 0, 1).  Substituting  these  points 
in  the  plane  equation  above  we  obtain  — ai  =  a3  =  1,  so  n“  can  be  expressed  as 

n“  :  U  =  -V  AotW  (9) 

OL  is  an  invariant  characterizing  the  plane  H®  in  the  invariant  coordinate  system. 

To  find  the  invariant  a,  we  eliminate  U  from  the  equations  of  the  two  cylinders  above, 
using  the  plane  equation  (9).  We  obtain  two  plane  conic  equations  in  the  variables  V^W: 

(flii  —  2di2  "i"  022)1^^  T  2q;( — Gji  -|-  d,x‘;^VW  -1-  oi^awW'^ 

-l-2( — dll  T  ^12  —  ®i3  A  023)1^  “t”  2Q:(dii  -|-  di3)W  -|-  dn  -1-  2di3  -b  d33  =  0 
d'liV^  +  2{-a[^a  -  a[2)VW  +  (a[^a^  +  2d;2a  +  022)^^ 

-t-2(— dji  —  -f  2((djj  -|-  +  ^23)^  d"  +  2®i3  +  ^33  ~  ^ 
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Since  these  equations  represent  the  same  conic  in  the  plane  11“,  they  must  have  the  same 
coefficients.  We  thus  obtain  five  equations  (from  the  five  conic  coefficients)  for  the  one 
unknown  a.  However,  we  have  already  seen  that  each  cylinder  has  only  one  independent 
coefficient,  so  only  one  of  these  equations  is  meaningful.  The  rest  are  either  identities  or  they 
are  dependent  on  the  first.  We  can  see  more  directly  that  the  3D  conic  in  H®  has  only  one 
independent  coefficient  in  our  invariant  coordinate  system.  The  conic  has  to  pass  through 
two  of  the  3D  reference  points  having  the  fixed  coordinates  (6),  and  be  tangent  there  to  the 
epipolar  planes,  which  are  known  with  fixed  values  from  the  corresponding  epipolar  lines. 
This  fixes  four  of  the  conic  coefficients. 

We  obtain  a  by  equating  the  coefficients  of  VW : 

On 

a  =  7 - ^ 

^11  ”  <^11  ^  ^12 

Substituting  a  in  either  of  the  conic  equations  above  will  give  us  the  description  of  the 
3D  conic.  All  the  coefficients  are  invariant,  but  only  one  is  independent,  as  discussed  above. 

The  other  3D  conic  can  be  found  in  a  similar  way.  Its  plane  H^  can  be  written  as 

U  =  V  +  ^W-^  (10) 

and  contains  the  reference  points  (1, 1, 1, 1),  (0, 0, 1, 1).  Substituting  this  in  the  cylinders 
arising  from  B'  we  obtain  the  conics 

(ill  +  2ii2  +  i>22)V^  +  2^(iii  d"  ii2)l^W  +  ^'^bnW^ 

+2{—bii0  —  ii2/5  +  ii3  +  b23)V  +  2[—bu/3^  +  ii3id)W  +  bii0^  —  2613/?  +  633  =  0 
iii ^  +  \^)VW  +  (i'li^d^  +  2b[^l3  + 

+2(— 6ii/9  +  b\^)V  +  2(— 611^^  +  iis/?  —  ^12/^  d-  ^23)^  d"  ~  26^3/?  +  633  =  0 

We  obtain  ^  by  equating  the  coefficients  for  VW : 

ill  d-  ii2  ■“  ill 

Substituting  this  in  the  conic  equations  above,  we  obtain  the  conic  in  the  plane  H**.  Again, 
all  the  coefficients  are  invariants,  with  only  one  being  independent. 
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In  sum,  each  conic  has  two  invariants  associated  with  it:  the  coordinate  of  the  plane 
on  which  it  lies  {a  or  0),  and  one  invariant  conic  coefficient  (or  any  one  function  of  the 
■>  coefficients).  Altogether,  we  thus  have  four  3D  invariants.  This  is  what  we  concluded  earlier 

from  a  simple  counting  argument. 

We  can  find  a  geometric  construction  of  the  conic  invariants  by  a  method  we  used  earlier. 
First  we  find  the  line  of  intersection  of  the  two  planes  11“,  11^.  In  each  plane  we  can  now 
draw  tangents  to  the  conics  that  are  parallel  to  this  intersection  line.  The  ratio  of  distances 
of  the  tangents  from  the  intersection  line  is  an  affine  invariant  of  the  conic. 

The  above  method  is  easily  applied  to  a  combination  of  a  conic  and  points.  Two  of  the 
invariant  reference  points  in  3D  will  be  determined  by  the  conic  as  before,  and  the  other  two 
will  be  the  given  points.  The  conic  is  reconstructed  using  the  same  equations  used  above 
(with  either  A  or  B).  The  system  will  have  two  3D  invariants:  the  plane  coordinate  a  (or 
/9)  and  the  invariant  of  the  conic  lying  in  it. 

4.  Conclusion 

We  have  reconstructed  3D  objects,  such  as  a  pair  of  non-coplanar  conics,  from  uncalibrated 
images,  up  to  an  affine  transformation.  No  knowledge  of  the  epipolar  geometry  was  needed; 
it  was  recovered  from  the  images.  The  method  is  based  on  using  invariants  of  epipolar  lines 
that  are  equal  in  the  two  images  when  these  lines  match.  This  invariance  is  used  to  find 
the  epipolar  geometry.  We  have  built  a  coordinate  system  which  is  common  to  3D  and  2D 
and  is  invariant  in  both.  This  invariant  system  has  made  the  reconstruction  very  simple  and 
general.  The  affine  invariants  of  the  3D  objects  are  obtained  very  easily  in  this  system.  Since 
many  curves  can  be  approximated  by  or  segmented  into  conics,  the  method  can  be  useful  in 
many  applications. 
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